HBR's 10 Must Reads 2023 by Harvard Business Review

Author:Harvard Business Review
Language: eng
Format: epub
Publisher: Harvard Business Review Press
Published: 2022-10-18T00:00:00+00:00

3. What is causing the waste and missed opportunities?

This question is usually the hardest, because it requires reexamining implicit assumptions about the firm’s current approach. To find the answer the firm must explore its data and get its subject matter experts and data scientists to collaborate. The focus should be on solving the alignment, asymmetry, and aggregation problems we identified earlier.

Addressing alignment. The goal here is to map the connections between AI predictions, decisions, and business outcomes. That requires thinking about hypothetical scenarios. We recommend that teams answer the following questions:

In an ideal world, what knowledge would you have that would fully eliminate waste and missed opportunities? Is your current prediction a good proxy for that?

If the telecom team members had answered the first question, they would have realized that if their AI predicted perfectly who could be won over by the retention offer (rather than who was about to leave), they could eliminate both waste (because they wouldn’t bother making offers to unpersuadable customers) and missed opportunities (because they’d reach every customer who was persuadable). While it is impossible to make perfect predictions in the real world, focusing on persuadability would still have led to great improvements.

After the ideal information is identified, the question becomes whether the data science team can make the required predictions with sufficient accuracy. It’s crucial that the marketing and data science teams answer this together; marketers often don’t know what can be done. Similarly, it is difficult for the data scientists to link their predictions to decisions if they don’t have subject matter expertise.

Does the output of your AI fully align with the business objective?

Remember the gaming company that used AI to identify features that would increase user engagement? Imagine the gains if the company had created AI that predicted user profitability instead.

A common mistake here is falsely believing that a correlation between the prediction and the business objective is enough. This thinking is flawed because correlation is not causation, so you might predict changes in something that correlates with profitability but does not in fact improve it. And even when there is causation, it may not map 100% to the objective, so your effort may not fully achieve your final outcome, leading to missed opportunities.

At the telecom company, asking this third question might lead the team to think not only about persuadable users but also about the increase or decrease in their profitability. A persuadable user with low expected profitability should have a lower priority than a persuadable user with high expected profitability.

Addressing asymmetry. Once you have a clear map that links the AI prediction with the decision and the business outcome, you need to quantify the potential costs of errors in the system. That entails asking, How much are we deviating from the business results we want, given that the AI’s output isn’t completely accurate?

At the telecom company, the cost of sending a retention promotion to a nonpersuadable customer (waste) is lower than the cost of losing a high-value customer who could have been persuaded by the offer (missed opportunity).

Download

Copyright Disclaimer:
This site does not store any files on its server. We only index and link to content provided by other sites. Please contact the content providers to delete copyright contents if any and email us, we'll remove relevant links or contents immediately.